I argue that at least one of the following propositions is true: the human species is very likely to become extinct before reaching a ’posthuman’ stage; any posthuman civilization is extremely unlikely to run a significant number of simulations of its evolutionary history ; we are almost certainly living in a computer simulation. It follows that the belief that there is a significant chance that we shall one day become posthumans who run ancestor-simulations is false, unless we are currently living (...) in a simulation. I discuss some consequences of this result. (shrink)

The human brain has some capabilities that the brains of other animals lack. It is to these distinctive capabilities that our species owes its dominant position. Other animals have stronger muscles or sharper claws, but we have cleverer brains. If machine brains one day come to surpass human brains in general intelligence, then this new superintelligence could become very powerful. As the fate of the gorillas now depends more on us humans than on the gorillas themselves, so the fate of (...) our species then would come to depend on the actions of the machine superintelligence. But we have one advantage: we get to make the first move. Will it be possible to construct a seed AI or otherwise to engineer initial conditions so as to make an intelligence explosion survivable? How could one achieve a controlled detonation? To get closer to an answer to this question, we must make our way through a fascinating landscape of topics and considerations. Read the book and learn about oracles, genies, singletons; about boxing methods, tripwires, and mind crime; about humanity's cosmic endowment and differential technological development; indirect normativity, instrumental convergence, whole brain emulation and technology couplings; Malthusian economics and dystopian evolution; artificial intelligence, and biological cognitive enhancement, and collective intelligence. (shrink)

For the last thirty years or so, there has been a search underway for a theory that can accommodate our intuitions in regard to moral duties to future generations. The object of this search has proved surprisingly elusive. The classical moral theories in the literature all have perplexing implications in this area. Classical Utilitarianism, for instance, implies that it could be better to expand a population even if everyone in the resulting population would be much worse off than in the (...) original. The main problem has been to find an adequate population theory, that is, a theory about the moral value of states of affairs where the number of people, the quality of their lives, and their identities may vary. Since, arguably, any reasonable moral theory has to take these aspects of possible states of affairs into account when determining the normative status of actions, the study of population theory is of general import for moral theory. A number of theories have been proposed in the literature that purport to avoid counterintuitive implications such as the one mentioned above. The suggestions are diverse: introducing novel ways of aggregating welfare into a measure of value, revising the notion of a life worth living, questioning the way we can compare and measure welfare, counting people’s welfare differently depending on the temporal location or the modal features of their lives, and challenging the logic of axiological and normative concepts. We investigate the concepts and assumptions involved in these theories as well as their implications for population theory. In our discussion, we propose a number of intuitively appealing and logically weak adequacy conditions for an acceptable population theory. Finally, we consider whether it is possible to find a theory that satisfies all of these conditions. We prove that no such theory exists. (shrink)

Because of accelerating technological progress, humankind may be rapidly approaching a critical phase in its career. In addition to well-known threats such as nuclear holocaust, the propects of radically transforming technologies like nanotech systems and machine intelligence present us with unprecedented opportunities and risks. Our future, and whether we will have a future at all, may well be determined by how we deal with these challenges. In the case of radically transforming technologies, a better understanding of the transition dynamics from (...) a human to a "posthuman" society is needed. Of particular importance is to know where the pitfalls are: the ways in which things could go terminally wrong. While we have had long exposure to various personal, local, and endurable global hazards, this paper analyzes a recently emerging category: that of existential risks. These are threats that could case our extinction or destroy the potential of Earth - originating intelligent life. Some of these threats are relatively well known while others, including some of the gravest, have gone almost unrecognized. Existential risks have a cluster of features that make ordinary risk management ineffective. A final section of this paper discusses several ethical and policy implications. A clearer understanding of the threat picture will enable us to formulate better strategies. (shrink)

A search is under way for a theory that can accommodate our intuitions in population axiology. The object of this search has proved elusive. This is not surprising since, as we shall see, any welfarist axiology that satisfies three reasonable conditions implies at least one of three counter-intuitive conclusions. I shall start by pointing out the failures in three recent attempts to construct an acceptable population axiology. I shall then present an impossibility theorem and conclude with a short discussion of (...) how it might be extended to pluralist axiologies, that is, axiologies that take more values than welfare into account. (shrink)

Suppose that A and B are two kinds of goods such that more of each is better than less. A is strongly superior to B if any amount of A is better than any amount of B. It is weakly superior to B if some amount of A is better than any amount of B. There are many examples of these relations in the literature, sometimes under the labels “higher goods” and “discontinuity.” The chapter gives a precise and generalized statement (...) of Strong and Weak Superiority and discusses different ways in which these relations can be relevant to the aggregation of welfare. It also proves a number of general results. One of the results gives rise to a dilemma: It can be used as an argument against the existence of value superiority or, alternatively, as an argument against the view that superiority entails a radical difference in value. (shrink)

_Anthropic Bias_ explores how to reason when you suspect that your evidence is biased by "observation selection effects"--that is, evidence that has been filtered by the precondition that there be some suitably positioned observer to "have" the evidence. This conundrum--sometimes alluded to as "the anthropic principle," "self-locating belief," or "indexical information"--turns out to be a surprisingly perplexing and intellectually stimulating challenge, one abounding with important implications for many areas in science and philosophy. There are the philosophical thought experiments and paradoxes: (...) the Doomsday Argument; Sleeping Beauty; the Presumptuous Philosopher; Adam & Eve; the Absent-Minded Driver; the Shooting Room. And there are the applications in contemporary science: cosmology ; evolutionary theory ; the problem of time's arrow ; quantum physics ; game-theory problems with imperfect recall ; even traffic analysis. _Anthropic Bias_ argues that the same principles are at work across all these domains. And it offers a synthesis: a mathematically explicit theory of observation selection effects that attempts to meet scientific needs while steering clear of philosophical paradox. (shrink)

The Doomsday argument purports to show that the risk of the human species going extinct soon has been systematically underestimated. This argument has something in common with controversial forms of reasoning in other areas, including: game theoretic problems with imperfect recall, the methodology of cosmology, the epistemology of indexical belief, and the debate over so-called fine-tuning arguments for the design hypothesis. The common denominator is a certain premiss: the Self-Sampling Assumption. We present two strands of argument in favor of this (...) assumption. Through a series of thought experiments we then investigate some bizarre prima facie consequences – backward causation, psychic powers, and an apparent conflict with the Principal Principle. (shrink)

To what extent should we use technological advances to try to make better human beings? Leading philosophers debate the possibility of enhancing human cognition, mood, personality, and physical performance, and controlling aging. Would this take us beyond the bounds of human nature? These are questions that need to be answered now.

Recently, in his Rolf Schock Prize Lecture, Derek Parfit has suggested a novel way of avoiding the Repugnant Conclusion by introducing what he calls “imprecision” in value comparisons. He suggests that in a range of important cases, populations of different sizes are only imprecisely comparable. Parfit suggests that this feature of value comparisons opens up a way of avoiding the Repugnant Conclusion without implying other counterintuitive conclusions, and thus solves one of the major challenges in ethics. In this article, I (...) try to clarify Parfit's proposal and evaluate whether it will help us with the paradoxes in population ethics. (shrink)

Population axiology concerns how to evaluate populations in regard to their goodness, that is, how to order populations by the relations “is better than” and “is as good as”. This field has been riddled with “paradoxes” which seem to show that our considered beliefs are inconsistent in cases where the number of people and their welfare varies. Already in Derek Parfit’s seminal contribution to the topic, an informal paradox — the Mere Addition Paradox — was presented and later contributions have (...) proved similar results.1 All of these contributions, however, have one thing in common: They all involve an adequacy condition that rules out Parfit’s Repugnant Conclusion: The Repugnant Conclusion: For any perfectly equal population with very high positive welfare, there is a population with very low positive welfare which is better, other things being equal.2 A number of theorists, however, have argued that we should accept the Repugnant Conclusion and hence that avoidance of this conclusion is not a convincing adequacy condition for a population axiology. Torbjörn Tännsjö, for example, argues that the Repugnant Conclusion is not at all repugnant but rather “an unsought, but acceptable, consequence of hedonistic utilitarianism”:3.. (shrink)

Can it be better or worse for a person to exist than not to exist at all? This old and challenging existential question has been raised anew in contemporary moral philosophy, mainly for two reasons. First, traditional “impersonal” ethical theories, such as utilitarianism, have counterintuitive implications in population ethics, for example, the repugnant conclusion. Second, it has seemed evident to many that an outcome can be better than another only if it is better for someone, and that only moral theories (...) that are in this sense “person affecting” can be correct. The implications of this Person-Affecting Restriction will differ radically, however, depending on which answer one gives to the existential question. The negative answer, which we argue against, would make the restriction quite untenable. Hence, many of the problems regarding our moral duties to future generations turn around the issue at hand. (shrink)

The human desire to acquire new capacities is as ancient as our species itself. We have always sought to expand the boundaries of our existence, be it socially, geographically, or mentally. There is a tendency in at least some individuals always to search for a way around every obstacle and limitation to human life and happiness.

The long sweep of human history has involved a continuing interaction between peoples' efforts to improve their well-being and the environment's stability to sustain those efforts. Throughout most of that history, the interactions between human development and the environment have been relatively simple and local affairs. But the complexity and scale of those interactions are increasing. What were once local incidents of pollution shared throughout a common watershed or air basin now involve multile nations - witness the concerns for acid (...) desposition in Europe and North America. What were once acute episodes of relatively reversible damage now affect multiple generations - witness the debates over disposal of chemical and radioactive wastes. (shrink)

In a recent paper in this journal, Ken Olum attempts to refute the Doomsday argument by appealing to the self-indication assumption, the idea that your very existence gives you reason to think that there are many observers. In contrast to earlier refutation attempts that use this strategy, Olum confronts and try to counter some of the objections that have been made against SIA. We argue that his defense of SIA is unsuccessful. This does not, however, mean that one has to (...) accept the Doomsday argument. A developed theory of observation selection effects shows why the Doomsday argument is inconclusive and how one can consistently reject both it and SIA. (shrink)

_Anthropic Bias_ explores how to reason when you suspect that your evidence is biased by "observation selection effects"--that is, evidence that has been filtered by the precondition that there be some suitably positioned observer to "have" the evidence. This conundrum--sometimes alluded to as "the anthropic principle," "self-locating belief," or "indexical information"--turns out to be a surprisingly perplexing and intellectually stimulating challenge, one abounding with important implications for many areas in science and philosophy. There are the philosophical thought experiments and paradoxes: (...) the Doomsday Argument; Sleeping Beauty; the Presumptuous Philosopher; Adam & Eve; the Absent-Minded Driver; the Shooting Room. And there are the applications in contemporary science: cosmology ; evolutionary theory ; the problem of time's arrow ; quantum physics ; game-theory problems with imperfect recall ; even traffic analysis. _Anthropic Bias_ argues that the same principles are at work across all these domains. And it offers a synthesis: a mathematically explicit theory of observation selection effects that attempts to meet scientific needs while steering clear of philosophical paradox. (shrink)

Unveränderter Nachdruck der Originalausgabe von 1930.

With very advanced technology, a very large population of people living happy lives could be sustained in the accessible region of the universe. For every year that development of such technologies and colonization of the universe is delayed, there is therefore a corresponding opportunity cost: a potential good, lives worth living, is not being realized. Given some plausible assumptions, this cost is extremely large. However, the lesson for standard utilitarians is not that we ought to maximize the pace of technological (...) development, but rather that we ought to maximize its safety, i.e. the probability that colonization will eventually occur. This goal has such high utility that standard utilitarians ought to focus all their efforts on it. Utilitarians of a ‘person-affecting’ stripe should accept a modified version of this conclusion. Some mixed ethical views, which combine utilitarian considerations with other criteria, will also be committed to a similar bottom line. (shrink)

Traditional ethical theories have paradoxical implications in regards to questions concerning procreation and our moral duties to future people. It has been suggested that the crux of the problem resides in an all too ‘impersonal’ axiology and that the problems of population axiology can be solved by adopting a ‘Person Affecting Restriction’ which in its slogan form states that an outcome can only be better than another if it is better for people. This move has been especially popular in the (...) context of medical ethics where many of the problems of population axiology are actualized. Examples are embryo or egg selection, pre-implantation genetic testing, assisted reproduction programmes, abortion, just to mention a few. I discuss a number of different interpretations of the Restriction and in particular one interpretation which I call Comparativism. According to this view, we should draw a distinction between uniquely and non-uniquely realizable people. The former people only exist in one out of two possible outcomes, whereas the latter exist in both of the compared outcomes. The idea is that we should give more weight to the well-being of non-uniquely realizable people or take it into account in a different way as compared to the well-being of uniquely realizable people. I argue that the different versions of the Person Affecting Restriction and Comparativism either have counterintuitive implications of their own or are compatible with traditional theories such as Utilitarianism. (shrink)

In some dark alley. . . Mugger: Hey, give me your wallet. Pascal: Why on Earth would I want to do that? Mugger: Otherwise I’ll shoot you. Pascal: But you don’t have a gun. Mugger: Oops! I knew I had forgotten something. Pascal: No wallet for you then. Have a nice evening. Mugger: Wait! Pascal: Sigh. Mugger: I’ve got a business proposition for you. . . . How about you give me your wallet now? In return, I promise to come (...) to your house tomorrow and give you double the value of what’s in the wallet. Not bad, eh? A 200% return on investment in 24 hours. (shrink)

Anthony Brueckner, in a recent article, proffers ‘a new way of thinking about Bostrom's Simulation Argument’ . His comments, however, misconstrue the argument; and some words of explanation are in order.The Simulation Argument purports to show, given some plausible assumptions, that at least one of three propositions is true . Roughly stated, these propositions are: almost all civilizations at our current level of development go extinct before reaching technological maturity; there is a strong convergence among technologically mature civilizations such that (...) almost all of them lose interest in creating ancestor-simulations; almost all people with our sorts of experiences live in computer simulations. I also argue that conditional on you should assign a very high credence to the proposition that you live in a computer simulation. However, pace Brueckner, I do not argue that we should believe that we are in simulation. 1 In fact, I believe that we are probably not simulated. The Simulation Argument purports to show only that, as well as , at least one of – is true; but it does not tell us which one.Brueckner also writes: " It is worth noting that one reason why Bostrom thinks that the number of Sims [computer-generated minds with experiences similar to those typical of normal, embodied humans living in a Sim-free early 21 st century world] will vastly outstrip the number of humans is that Sims ‘will run their … ". (shrink)

Once upon a time, the planet was tyrannized by a giant dragon. The dragon stood taller than the largest cathedral, and it was covered with thick black scales. Its red eyes glowed with hate, and from its terrible jaws flowed an incessant stream of evil-smelling yellowishgreen slime. It demanded from humankind a blood-curdling tribute: to satisfy its enormous appetite, ten thousand men and women had to be delivered every evening at the onset of dark to the foot of the mountain (...) where the dragontyrant lived. Sometimes the dragon would devour these unfortunate souls upon arrival; sometimes again it would lock them up in the mountain where they would wither away for months or years before eventually being consumed. (shrink)

In a recent paper in this journal, Ken Olum attempts to refute the doomsday argument by appealing to the self–indication assumption (SIA) that your very existence gives you reason to think that there are many observers. Unlike earlier users of this strategy, Olum tries to counter objections that have been made against (SIA). We argue that his defence of (SIA) is unsuccessful. This does not, however, mean that one has to accept the doomsday argument (or the other counter–intuitive results that (...) flow from related thought–experiments). A developed theory of observation selection effects shows why the doomsday argument is inconclusive, and how one can consistently reject both it and (SIA). (shrink)

It is customarily assumed that paracomplete and paraconsistent solutions to liar paradoxes require a logical system weaker than classical logic. That is, if a logic is not fragile to liar paradoxes, it must be logically weaker than classical logic. Defenders of classical logic argue that the losses of weakening it outweigh the gains. Advocates of paracomplete and paraconsistent solutions disagree. We articulate the notion of fragility with respect to the liar paradox and show that it can be disentangled from logical (...) strength. Strength and resilience to paradox do not force a trade-off with respect to liars: there can be logics which are not weaker than classical logic and are solid to the liar. (shrink)

The person-affecting restriction, in its slogan form, states that an outcome can be better than another only if it is better for someone. It has a strong intuitive appeal and several theorists have suggested that it avoids certain counterintuitive implications in population ethics. At the same time, the restriction has highly counterintuitive implications and yields non-transitive orderings in some nonidentity cases. Many theorists have taken this criticism to be decisive. Recently, however, there have been some reformulations of the restriction, suggesting (...) that the restriction survives this “old” criticism. This paper investigates the viability of those reformulations, which are versions of “Comparativism,” and argues that most of them either have counterintuitive implications or are extensionally equivalent with impersonal theories, but that “Soft Comparativism” seems to have an advantage over impersonal theories. (shrink)

This paper focuses on the relations between population ethics and metaethics. Population ethics gives rise to well-known paradoxes, such as the paradox of mere addition. After presenting a version of this paradox, it is argued that a different way to dismantle it might be by considering it as a way to change our standard view of justification in moral theory. Two possible views are considered: a non-cognitivist approach to justification and to the explanation of inconsistency in morals; Parfit's suggestion that (...) certain paradoxes might be «quarantined» without shaking our confidence in moral theories encapsulating them. (shrink)

Positions on the ethics of human enhancement technologies can be (crudely) characterized as ranging from transhumanism to bioconservatism. Transhumanists believe that human enhancement technologies should be made widely available, that individuals should have broad discretion over which of these technologies to apply to themselves, and that parents should normally have the right to choose enhancements for their children-to-be. Bioconservatives (whose ranks include such diverse writers as Leon Kass, Francis Fukuyama, George Annas, Wesley Smith, Jeremy Rifkin, and Bill McKibben) are generally (...) opposed to the use of technology to modify human nature. A central idea in bioconservativism is that human enhancement technologies will undermine our human dignity. To forestall a slide down the slippery slope towards an ultimately debased ‘posthuman’ state, bioconservatives often argue for broad bans on otherwise promising human enhancements. This paper distinguishes two common fears about the posthuman and argues for the importance of a concept of dignity that is inclusive enough to also apply to many possible posthuman beings. Recognizing the possibility of posthuman dignity undercuts an important objection against human enhancement and removes a distortive double standard from our field of moral vision. (shrink)

Evolutionary development is sometimes thought of as exhibiting an inexorable trend towards higher, more complex, and normatively worthwhile forms of life. This paper explores some dystopian scenarios where freewheeling evolutionary developments, while continuing to produce complex and intelligent forms of organization, lead to the gradual elimination of all forms of being that we care about. We then consider how such catastrophic outcomes could be avoided and argue that under certain conditions the only possible remedy would be a globally coordinated policy (...) to control human evolution by modifying the fitness function of future intelligent life forms. (shrink)